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Several sets of data are analyEed^ all of niich 
damonstrate that estimates of achievement rates can differ 
dramatically when different time periods are used for an evaluation 
of a compensatory education program* Specif ically^ it is found that 
inaluding the summer months in an evaluation often considerably 
reduces estimates of achievement and hence monthly aAhievement rates* 
In other words^ achievement gains made during the school year are not 
sustained^ even until the next fall, These findings are consistently 
supported by the data regarflless pf the standairdized achievement test 
used^ the grade levels the subject area ^ or the progra©^ 
evaluation of a one--year program should at least take account of 
fall^to-fall achievement* For multi-^year programs^ evaluations should 
be based on a tima period that includes the summer follotiing each 
year of the program* including the folloMing summer in an evaluation 
will allow tha conclusion to refleGt the ejttent to which student 
aciiievement is sustained and will^ therefore, provide more useful 
information to those concerned with improving programs, (BC) 
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A Policy Research Report is an officiQl document of the Educational Policy 
Research Center. It presents results of work directed toward specific research 
objuctivo. The report is a wuniprehensive trcatmeni of the objegtives, scope, 
methodology, data, analyses, and conclusions, and presents the background, 
practiual ^Significance, and technical infomiation required for a complete and 
ftill understanding of the research activity. The report is designed to be directly 
useful to educational policy makers, 

RiSEARCH MiMORANDUM 

A Research Memorandum is a working paper that presents the results of work 
in prcigress. The purpose of the Research Memorandum is to invite commef^t on 
research in progress. It ss a comprehensive treaim^ffnt of a single research area 
jr of a facet of a research area within a laner field of study. The Memorandum 
presents the background, objectives, scope, summary, and conclusions, as well 
as method and approach , in a condensed form. Since it presents views and con- 
ciusions drawn during the progress of research activity, it rnay be expanded or 
modified in the light of further research, 

RESEARCH NOTE 

A Research Note is a working paper that presents the results of study related to 
a single phase or factor of a research problem. It also may present preliminary 
exploratlon^of an educational policy issue or an interim report which may later 
appear as a larger study. The purpose of the Research Note is to instigate dis- 
cussion and criliwisnu It presents the concepts, nndings, and/or conclusions of 
the author. It may be altered, expanded, or withdrawn at any time. 
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EXEOOTrVE sm^RY 



Support for compensatory education rests ou the assumptton that 
grcaf;er achievement can improva the acaidem& fOTure of disadr^antaged 
Btudent^i and that a better academic ftJture can improve their Itle 
chances* In order to change students* futures^, an increase in aehieve^ 
ment from compensatory education should persist in the years following 
a student*s participation in compensatory educatlan programs, Thereforej 
evaluations of coinpensatory education should include measures of long-term 
or sustainisd, achievement* At the very leasts these measures should 
reflect the extent to which incireasea in achlevemeat_persisc over the 
ijummer following a school-year program, 

Eva luat ions of compensatory education such as those of ESI A Title 1 
have generally been based on tests administered in the fall and apring 
of a given school years ra'cher than on tests measuring austained achieve- 
ment. The findings of SRI's study of Title I evaluations and research 
on achievement over the summer suggest that high gains of disadvantaged 
students during the school year might be followed by large losses during 
the summer* ThuSj evaluations reflecting the extent to which gains are 
sustained over the summer could produce conclusions about the success of 
a program very different from those based on measures of fall-to-spring 
achievement. 

To test the hypothesis that the period of time on which an evalua- 
tion is based can significantly alter the conclusiona reached, we obtained 
data from evaluations of several compensatory education programs. We 
analyzed these data by calculating estimates of achievement rates over 
different time periods and comparing the results. 



T, Thomas and S. Pelavln^ ''Patterns in ESEA Title I Reading Achievement^" 
EPRC 4537^12s Stanford Research Institute^ Menlo Park^ California 
CMarch 1976). 
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Our analyses demonstrace that estimaCes of achievement rates differ 

dramatically when different time periods; are used. Specif leallyi our 
results show that including the summer months in an evaluation can consider 

ably reduce astiraates of achievement. Large increases in achool-year 

achtevement are not sustained even until the next fall. The findings 

are remarkably consistent across achievenient tests, grade levelSj subject 
areas, and programs. 

We conclude that if compansstory aducation prdgrams dra going to be 
evaluated on ths basis of student achievements It Is essential tor have 
measures of sustained achievement. Thus, an evaluation of a one-year 
program should at least take account of fall-to-fall achievement. For 
multi-year programs* we suggest that evaluations be based on a time period 
that Includes the suiraner following each year of the program. Including 
the following summer in an evaluation will allow the conclusions to reflect 
the extent to which student achievement is sustained and will, therefore^ 
provide more useful information to those concerned with improving programs. 

School districts should administer achievement tests at least every 
fall and preferably every fall and spring* These data would show the 
extent to which school-year gains are sustained through the following 
summer* Alsos the extent and causes of sunmer losses should be explored. 
Since there are no simple solutions (for examples little research exists 
to show that summer school would alleviate - the suirsner losses) ^ it is 
Important to be able to determine why the losses oeeur in order to develop 
appropriate remedies. 
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I INTRODUCTION 



A fundamental assumption underlying compensatory education is that 
greater achievement can Improva the academic future of disadvantaged 
students/and that this in turn can improve their ^'lifa chances." There- 
fore, one of the goals of Title I of the Elementary and Secondary Educa- 
tion Act (ESEA) la to increase the achievement- of disadvantaged students. 
In order to change students' futuresj this increase in achievement should 
be reflected In the years following students' participation in compensa-^ 
tory education programs s At a minimum, an increase In achievement should 
persist over the summer following a school-year program, Howeverj evalu- 
ations of compensatory education in general and of ESEA Title I in partic- 
ular have not Included estimates of sustained achievement. Instead, they 
have been based on tests administered in the fall (pretest) and spring 
(posttest)j and have used student achievement during this period^ that 
is, the pdsttest score adjusted in some way for the pretest score, to 
judge the success of the program. 

We hypothesised ;tho^t evaluations based on measures of sustained 
achievement would lead to different conclusions than evaluations based 
on fall-to-spring achievement. To test this hypothesis, we obtained 
data from evaluations of several compensatory education progrmns, With 
these dataj we demonstrate that fall and spring measures, by themselves, 
are insufficient indicators of program success and may even be misleading. 
Our conclusion is that evaluations of achievement in compensatory programs 
should be based on a time frame that allows estimates to be made of sus- 
tained achie' nant, . - 

In addition to an appropriate time frame, one must have a procedure 
that- can distinguish the portion of the achievement gain that is attri- 
butable to the progrOT* There are aV variety of techniques, each with 
its flaws, for estimating the program's effect on achievement. Arguments 
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over the best mechodj pafticularly in evaluations that/rely on standard- ■ 
Ized achievement. ttsts, ^ilT continue. for years to come. We do not intend 
to argue the relative merits of these methods in this paper* - Rather, our ; 
purpose is to demons Crate jthat regardless of how^^achlevement ^ls measured r^^^^ 
the time frame used in the evaluation can dramatically affect the results. 
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II BAC^HOUND 



This work was motivated by a study-of ESEA Title I achievement dona 
by the Educ Policy Research Center (EPRC) and relates directly to 

>that study. : In 7'Pattema in ESEA Title I Reading Achievement^" Thomas 
and Pelavin (1976) analysed data from 283 State Educational Agency (SEA) 
Title I evaluation reports for the academic years 1968-69 through 1973- - 
74, For those states reporting adequate dataj average monthly reading 
achiev ement.was calculated* Th^se analyses showed that average 'monthly 
reading achievement across states was close to 1, 1 grade -equivalent 
months.^ That Is^ Title I participants appeared, on the average^ to be 
gaining more than one grade-equivalent month for each month of the: school 
year. In terms of the imof f iclal standard of successy \^ich is a month's 
gain for each month in the program, ESEA Title I appeared to be a success. 

To corroborate this finding, the report looked for evidence of a 
Title I impact in the results of three statewide testing programs. Each 
of the three states had reported that Title I students were achieving at 
a monthly-=.rate^.greater than 1.0. The .expectation was that there would 
be an upward shift in . the statewide test results over the years for low- 
percentile students (those most likely to be in Title. I)» The data from 
the statewide tests did not reflect such a shifty and hence did not^ pro- 
vide evidence^to support the apparent success of Title I as found from 
the SEA data. ^ T 

The EPRC report offered a number of possible explanations for these 
apparently contradictory findings. Since the most plausible explanations 
involved the amount of achievenient gained or lost during the summer months 
a review of the relevant research was performed. The review suggested 
that disadvantaged students generally achieve at a much slower rate 
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during the summer than during the school year* In factj during the 
summer iLonthij many dls^Jvantaged students show no achievement gain 
and even suffer a loss* If this is frequently true for ESEA Title I 
participantSi measuring achievement over- a full calendar year would . - ^ 
produce a smaller monthly rate than measuring achievement from a fall 
pretest to a spring posttest as in the SEA reports* 

In view of the long-range goals of compensatory education, if 
gains made over the school year are not sustained even over the sunmier, 
program evaluations should reflect this phenomenon. The EPRC report 
pointed out that such speculation could be confirmed only by analyzing 
longitudinal data on individual students* We have obtained such data 
and will demonstrate in this paper how conclusions about program effec- 
tiveness change when sustained achievement is measured. i- 



The review Includes studies based on 
and without school -year compensatory 
is with the pattern of growth of the 
defined to achieve at a rate of nine 
school year and one month over the s^ 



disadvantaged students both with 
education programs. The contrast 
50th percentile student, who is 
grade-equivalent months over the 
jnmer, - 



- - - ^ - III METHOD OF ANALYSIS - r - _ 

V. SEA Title I evaluations almost eKclusiv^ly report achievement rates 
based on data from a fall pretest and a spring posttest* Thereforej Che 
evaluation reports reflect achievement during a relatively short period 
of time J usually seven months** For the reasons just presented, we are 
particularly interested in whether or not a program's effect peidsists 
over the summer* To determine the extent to which the effect on achieve- 
ment Is sustained until the beginning of tha next school year, data from 
an additional test administered In che fall of the subsequent school 
year were used to calculate both the fall -to-sprlng and the fall-to-fall 
ratesr of avsh lavement. - Comparing e/ich of these rates to the unofficial 
Title I standard of success --a month-f or-month achievement rate^^ we estab 
lished the extent to which conclusions change about the success of a 
program when . the longer period of time is used. 

ihe following example illustrates our analytical approach for one- 
and two-^year evaluations. The data consist of a matched, longitudinal 
sample of 87 students who have participated In a compensatory education 
program during Grades 3 and 4. ; 

To evaluate one year of the program^ we use grade-^equlvalent means 
for three administrations of the Reading Comprahension portion of the 
Gates-MacGlnltie Reading Test. The means for the beginning and end of 
Grade 3 (3 Fall and 3 Spring) and the beginning of Grade 4 (4 Fall) arai 

3 Fall 3 Spring 4 Fail : - 
2.57 3.62 3.00 



The pretest to posttest interval does vary from four to eight months. 
However, the average and by far the most coiranon is seven. 

Sea Table 3, line 1^ in the next section. " ' • " r ^ 



Mean achievement from fall Co spring (3 Spring minus 3 Fall) Is approxi* 
matelylO months. Dlvidini this iain by the number of months batwaen 
test ■administrations (approximately seven months) j we get a monthly 
achievement rate' of about .1,5 months per month. Compared to the month- .. 
for^month standardj the program would be judged successful on this basis. 
However^ the mean achievement gain from fall to fall (4 Fall minus 3 Fall) 
is only four months^ which translates into a mOTthly rata of 0.4 month 
per monthp^ During this longer time interval the program would not be 
judged a success* 



: In this eKamples we calculated rates of achievement for both 7-^ ■ 
month and 12 -month periods and used the month -^for-'month standard as the 
reference point for success/ Month-for-month achievement is the expected 
rate, by definition * for the average or 50th percentile student during - 
the school year. Though there is not an esCabllshed monthly rate of 

achievement for' the disadvantaged students the annual rate of seven 

■ ' . _ • • . . . - + • ■ - ^- - - 

months per year, is frequently used* and Is supported by national Tittle I 

data. Therefore we prefer to use an expected annual achievement rate 

of seven months gain as our point of comparison rather than a monthly 

rate of achievement. In our, example^ the annual achievement of four 

months clearly does not exceed the expected seven months and suggests 

that high achievement_during the pretest^to-^posttes t period is not even 

susjtalned until the beginning of the next school year* , 



A grade-equivalent'-year, by definitions consists of 10 grade -equivalent 
months, Thereforej to calculate a monthly rate in grade equivalents, 
the annual gain is divided by 10. 

The office of Education uses this estimate. Sees for example, the 1975 
U,S/ General Accounting Of ficeVs "Report to the Congress by the Control- 
ler General of the United States, Assessment of Reading Activities Funded 
Under the Federal Progrm of Aid for Educationally Deprived Children,*' 

T, Thomas and S. Pelavln, op- cit. , have calculated annual achievement 
by dividing p^retest Cfall) scores for large samples of disadvantaged 
students by the grade level (the number of years the students had been 
in school). For all grades, these annual rates were close to seven 
months, Thes^ samples included some students who had previously partici-- 
pated in Title 1, hence the seven months is conceivably an overestimate 
of annual achievement. 



To illustrgte a two-year evaluation,- the example is extended -through 
a aecond year of the program. To the threa grade*equivalent means used 
abov&j we adde< the results of two more administratians of the Reading 
Comprehension subtest. The means for the beginning and end of. Grade ::3 
(3 Fall and 3 Spring), the beginning and end of Grade 4 (4. Fall and 4 
Spring), and the beginning of Grade 5 (S Fall) are! 



3 Fall 3 Spring k Fall 4 Spring 5 Fall 
2.57 3.62 3.00 4, 12 3.53 



We use these five means to demonstrate ho^ estimates of two-year achieve** 
ment rates, differ under three time periods* The. shaded areas in Figure I 
illustrate the three different time periodsr" ^ 



3 FALL 



3 SPRING 



4 FALL 



4 iPRING 



i FALL 



FtRiT iSTlMATi 

CAii 1 














iiCOND ESTIMATE 
CASi 2 




















THIRD ESTIMATE 

cAf i 3 ; 











FIGURE t THREE POSSl'BLE TIME PERIODS FOR A TWO-YIAR EVALUATION 



First, we consider a two-year evaluation that estimates a monthly 
achievement rate for ^eaeh of the pretest (fall) to post test '(spxing) 
intervals. This is the procedure used in most multi-year Title I evalu- 
ations* We have already shown that^ the achievement rate from 3 Fall to 
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3 Spring is 1*5 months per month. Achievement for the second year (Grade 
/i>„ii approximately 12 months (4 Spring minus 4 Fall), which yields an 
achievement rate of 1,6 months per month. If the program were judged 
on the basis of these two rates j each of which exceeds the month-for- 
month standard J it would be considered a success* 

A second estimate of the achievement rate over two years is obtained 
by calculating the achievement from 3 Fall to 4 Spring, thus Including 
the intervening summer somths* The achievement during this period 
(4 Spring minus 3 Fall) is 15*5 monthSv To^ obtain a monthly rate, th^ 
achievement is divided by the number of months in the interval' 10 
months from 3 Fall to 4 Fall +7 months from 4 Fall to 4 Spring ^17 ~ 
months* This procedura yields an achievement rate of 0* 9 month per 
month* While this rate Is considerably lower than the pretest- to^posttest 
rate for either Grade 3 or 4| it is nonetheleas close to the month-*for- 
month standard* Hence the program might still be Judged successful* 

The third time period spans two full calendar years. We subtract 
th^ Grade 3 fall test from the Grade 5 fall test (5 Fall ffllnus 3 Fall) 
and divide this gain of 9,6 months by 20 months (10 months for each of 
the two years)* This gives an achievement rate of 0*5 month per month* 
Clearly^ for this time period the program would not be considered a suc- 
cess* A inonthly gain of 0,5 is far below the month-for^month standard, 
and average achievement of five months per year is below even that expected 
for disadvantaged students. 

Our conclusion is that the third time period is the appropriate one 
since it reflects the extent to which program gains persist until the 
beginning of the following school year. Since the gains arer-conslderably 
smaller under this time period, we conclude that high gains achieved 
during the school year are not sustained over the sunnner months* 

The following section describes our samples and presents the data 
to support these conclusions* ^ The data prBsented are all in the metric 
of grade equivalents* We are fully cognizant of the weaknesses of the 



grade^equlvalent scale, particularly for measuring growth,* Nevertheless 
we wanted to be consistent with previous discussions of the impact of 
compansatory education programi on achievement. These discussions have 
reliad heavily on -the grade-equivalent metric because It provides a means 
of aggregating and Gomparing different tests. As a result, both the 
standard for success and eKpectations for the disadvantaged student are 
expressed in terms of grade equivalents* However, we have carried out 
analyses demonstrating that the patterns of interest are not seriously 
affected by the metric. Since the transformations are monotonic and 
close to linear except at the extremes, this finding Is not surprising. 
Appendix A presents analyses comparing grade equivalents , standard scores 
and percentiles. 



See, for example, Angoff (1971), Cronbach (1970), Harcourt Brace 
Jovanbvich (1973), or Tallmadge (1975). 



IV RESULTS 



To compare jud^ents of program success under dlfforent time periods, 
obtained multi-year data from two major sources i the evaluation of a : 
large mldwestern city's compensatory education program and the evaluations 
of four junior high ichool compensatory education programs in California* 
All the programs span several grades and collect data annually in both 
fall and spring* All the data had been collected originally as part of: 
annual evaluations and were therefore cross-sectional by year. 

. By matching students across time and test administrations^ we were 
able to create longitudinal data that pemltted us to compare the same 
group of students during different time periods* The use of only matched, 
longitudinal samples itnpiies that whenever two means are compared , they 
are based^on data from exactly the same students. We present these data 
below/ We describe first the sample and results from Clcy M, the mld^j 
western city, and then the samples and results from California. 

City M •■ ■ ^ : : ^ . 

City M has a citywlde Title I reading program for eligible students 
In Grades 3 through 9* The Reading Comprehension section of either 
ievel C or: D of the Gates-MacGlnitie Reading, Test was administered to 
the students In both fall and spring of each school year. We received 
data for six cohorts of students In the progr™ during the school years 
1971-72 through 1974-75, including both public and nonpublic school stu- 
dents* The scores are reported only In individual grade equlvalentSj so 
all our analyses use this metric. 



The. level administered was determined by the student's instructional , 
level rather than grade level. ^ 

We use "cohort" to refer to a group of students who progress through 

— schooI_together*^fI^ : : : - - 



Our data_can be presented in a number of different ways since Chey-r 
Include^several cohorts and grades as well as both public and nonpublic 
school students* For simplicity, we present in this section only samples 
drawn from the larger set of public school students* The corresponding 
data for the nonpublic school students are contained in Appendix D* ^ 

■For each grade, we defined two samples of public school students. 
The first sample consists of students who were tested at least three * 
consecutive times: in fall and in spring of one academic year and in 
fall of the next year* These students have participated In the^reading^^ 
program for at least the one school year between the fall and spring 
tests* The second sample is a subset of the first and consists of all 
students who were tested at least five consecutive times: In fall and 
in spring of two consecutive academic years and in the following fall* 
These students have participated in the program for at least the two , 
school years spanned by the test points* Defining the sMiiples in this 
way allows each mean for a given sample to be based on the same students. 

Since we included in our samples only those students with sufficient 
test pointSs the samples do not reflect all students in the program for 
a given year and grade* However, for students tested In a given grade, 
we have compared each mean from our matchedp longitudinal sample to each 
mean from a larger sample of one-year participants* The means of the 
one-year participants were consistently higher. However, since the dif- 
ferences were always small and the pattern of gains quite similar, our 
conclusions were not affected. (See Appendix B for a comparison of the 
longltudtnal and the one-year participant samples*) 

In the following tables, we .present data by grade for the two samples 
of public school students (those with three and five consecutive test 
administrations). These data are presented separately for each cohort, 
by' grade, in Appendix C, With very few exceptions , the patterns described 
below are consistent across cohorts, grades, and both public and nonpublic 
school students,^ 



"We have eliinlnated means based on fewer than 30 students. 



; The grade-equivalent means and standard davlations for the samples 
with three consecutive test points are presented by grade level in - 
Table 1. Hencej for Grade 3 we show the means and standard deviations 
for - the fall and spring of Grade 3 and the^f all of Grade 4. -^The^ sample : 
siEes for the grades range from 128 to 980. For these samples. Table 2 
presents achievement gains and rates In grade-equivalent months based 
on the means in Table 1* Colmnn I contains the differences between 
the fall and spring scoresj and Column II shows the annual (fall-^to-f all) 
gains. Column III presents the fall-^to-spring gains as monthly achieve- 
ment rates (Column I divided by 7i the average interval between fall and 
spring tests)* Column IV expresses the fall-to-^fall gains in terms of 
monthly r.ites CColumn II divided by 10)* 

Table 1 : ■ . 

CITY M MEANS AND STMDARD DEVIATIONS IN GRADE EQUIVAI .OTS 
FOR THE GATES -MacGINltlE READING T^ BY GRADE 
FOR PUBLIC SCHOOL STUDENTS WITH AT LEAST THREE CONSECUTIVE TEST POINTS 













Grade 


N 


Fall 


■ Spring 


Fall 


3 


272 


2.23 


3.29 


2.78 






(1.04) 


(1.42) 


(0.96) 


4 


931 


2.65 


3.58 


3.18 






(0.83) 


(1.19) 


(0.96) 












5 : 


980 


3.26 


4.30 


4.01 






(0.99) 


(1.38) 


(1.30) 


6 


316 


3.85 


4.78 


4.42 






(1.2) 


(1.47) 


(1.32) 


7 


128 


4.35 


5.25 


4.95 - 






(1.24) 


(1.68) 


(1.41) 



Comparing Columns I and II M^^T see that^ In each grades 

the achlevemant gain Is substantially lower when based on the fall-to- 
fall period^ (Column IX) than when based on the traditional fall-to-spring 



period (Column 1). when the f all-to-iprlng achievement gains are trans- 
lated into monthly achievement rates (Column III), all the samples have 
considerably higher rates than the month-for-mon th^ standard, which s 
gests a highly auccessful program* - . . 

^ . . . - . Table :2- - 

CITY M ACHIEVEMEOT GAINS MD MONTHLY RATES BASED ON TOO 
DIFFERENT PERIODS OF TmE FOR PUBLIC SCHOOL STUDENTS 
WITH AT LEAST THREE CONSECUTIVE TEST POINTS 



Monthly Achievement Rates 
III IV 
Fall to Spring Fall to Fall 
. (I ^ 7) (II ^ 10) 



3 


272 


• 10.6 


5.5 


1.5 


0.6 


4 = 


931 


9.3 


5.3 




0.5 


5 


980 


10.4 


7.5 


\ 1.5 - 


■ 0.8 


6 


316 


9.3 


5.7 


1.3 


0.6 


7 


128 


9.0 


6.0 


-..—"-^ 1.3 


0.6 



The achievement is based on the means in Table 1* 



However, the monthly achievement rates in Column IV that are based 
on the fall-to-fall period present a very different picture . These rates 
are all much lower than the month-for-month standard. In fact, with one 
exception, these rates do not reach the seven months expected for dis- 
advantaged students. In the one exception (Grade 5) this expectation 
is exceeded only slightly* Hence, while the traditional fall-to-spring 
evaluation periods suggest a highly successful program, the more appro- 
priate fall-to-fall Interval suggests an unsuccessful program, one that 
does not produce sustained achievement, - 



Achievement in Grade- 

jquiva lent Months^ 

I " ^ -II 
Grade N Fall to Spring Fall to Fall 



13 



We now extend our presentation to a two-yaar evaluation. . Tables 3 
and 4 display data for the group of students with five consecutive test 
points; Table 3 presents, the grade-equivalent means and standard devia- 
tions for the five test points, with sample sizes ranging from 45 to 324 

. ; ■ ' - '.Table' 3 . • ' , / 

CITY M MEANS AND STMDABD DEVIATIONS IN GRADE EQUIVALENTS FOR THE 
GATES-MacGINITIE READING TEST BY GRADE FOR PUBLIC SCHOOL STUDENTS 
' / ' WITH AT LEAST FIVE CONSECUTIVE TEST POINTS 



Grade 


_N_ 


1 

Pall 1 


II 

SprlnR 1 


III 
Fall 2 


IV ' 

SprlnR 2 


V 

Fall 3 




87 


2.57 
(1.49) 


3.62 
■ (1.77) 


3.00 
(1.10) 


4.12 
(1.42) 


3.53 
(0.92) 


4-5 

..... 


■ 324 


2.75 
(6.94) 


3 . 54 
(1.28)- 


3.11 
(0.88) 


4.27 
(1.24) 


3.87 
(1. 15.) 


5-6 


130 


3.22 
(1.15) 


4.14 ; 

(1.47) 


3.86 ' 
(1.22) 


4.81 
(1.51) 


^.41 
(1.25) 


6-7 


45 


4.08 
(1.19) 


4.91 
(1.58) 


4.52 
(1.07) 


5.44 
(1.47) 


5.04 
(1.25) 



Table 4 shows the monthly achievement rates for two-year evaluations 
based on three different time periods (see Figure 1), Columns lA.and 
IB present the rates for each pretest (fall) to posttest (spring) Inter- 
val. In each grade, the monthly rate for each year equals or exceeds 
the month-for-month standard, A program that meets or exceeds the 
month-for-month standard for two consecutive years would undoubtedly 
be considered quite successful. - ^ 1: 

; ; Column II presents the monthly achievement rate based^hn . the 
from the beginning of the first year (Fall I) to the end of the second 
year (Spring 2). This estimate includes the losses incurred during the 
Intervening summer. While the rates are lower than those obtained for 
each schopl year separately, they are still close to the mon th-f or-month 
Standard :and„ w«3U Id ^likely lead to the conclusion that: t^^ is_a _ ^ 

success, - ■ . ' 



Table 4 



CITY M TWO-YEAR MONTHLY ACHIEVmENT RATES BASED ON THREE 
DIFFERENT PERIODS OF TIME FOR PUBLIC SCHOOL STUDENTS 
: WITH AT LEAST- FIVE CONSECUTIVE -iTEST POINTS^ 
(Grada-Equivalent Metric). 



II 



III 



Grade 

4- 5 

5- 6 

6- 7 



N 

87 
324 
130 

45 



A 

Year 1 
(Fall 1 to 
Bprlng 1) 

1.5 

1,3 

1.2 



Year 2 
(Fall 2 to 
Spring 2) 

1.6 
1.7 

- 1.4 \ 
1,3 



Fall 1 to 
Spring 2 

0,9 
0. 9 

0.9 
0.8 



Fall 1 to 

■Fall 3 



0.5 
0,6 
0, 6 
0.5 



The rates are calculated by dividing the achievement gain for each 
interval (baaed on the means in Table-3) by the number of grade- 
equivalent months in the interval (7 for Columns lA'and IB^ 17 for 
Column II, and 20 for Column III). / 



Column III presents the average monthly. achievement rate based on 
the full two calendar years, from the beginning of the first year (Fall 
1) to the beginning of the third year (Fall 3). This estimate includes 
both the intervening summer and the ~suimner following the second year. 
We see under this time period that the achievement rates are substantially 
lower.: In all four„gradeSs the'two-year rate under this __tiine frame is : 
considerably less than the month-for-month standard. If we CQmpare the 
average annual achievement- to the expectation of seven monthsi none of 
the grades exceeds that expectation. Under this time framej the program 
would not be judged a success. 



The average annual achievement in grade-equivalent months for the four 
grades is 4.75 S.S, 6,Ds-and 4.8, respectively. These are calculated 
by :dlviding the two-year gain by 2V 



while the third time period, Fall 1 to Fall 3, produces a far more 
discouragini picture Chan that produced by the other two methods, we 
believe that this is the appropriate time frame for a two-year evaluation. 
This period ts the only one that captures the extent to which achievement 
is sustained over the summer months. Since these rates are considerably 
smaller than those based on periods excluding one or both summer Intervals 
we conclude that high gains achieved during the school year do not persist 
until even the beginning of the next school year. 

Cal Ifornia 

Our second source of data was California fitate Damonstration Programs 
in Intensive Instruction in Reading and Mathematics. These schoolwide 
programs are limited to Junior high schools located in lo\c-'incorne areas. 
Participants are predominantly educationally disadvantaged students who 
meet eligibility requirements for Title I The programs began in Grade 7 
during the 1969-70. academic years moved with the students to Grade 8 
in I97O-7I5 and to Grade 9 in 1971-72. In 1972-735 the three-year cycle 
began again. ■Since the programs are schoolwidei there can be no "gradu- 
atlon" out of„the._program; all__students rematn_in the program j^qr all 
three years / 

In some distridtSi other compensatory funds were used to replicate 
the program in those grades not supported by the state* The data we 
obtained are from evaluations of both original state defflonstratlon pro- 
grams and district replications. The data are for one cohort of students 
in each of three reading programs (one school in each of three cttlos) 
and for three cohorts of students in a mathematics program (two cohorts 
in one school and one cohort in a second school ^ all in the same city) » 
Hence we have six different samples.' For all the programs, the Compre-. 
hensive Test of Basic Skills (CTBS) was administered in the fall and 
spring of Grades 7, 8, and 9, with the reading portion administered in 
the reading programs and the mathematics portion in the mathematics pro- 
gram. The forms and levels of the CTBS In each sample are shown in Table 5 



Table 5 



THE FORMS AND LEVELS OF THE CO^EPREHMSIVE TEST OF BASIC SKILLS 
ADMINISTERED TO EACH Si^ffLE 



Mathematlcgs City A \ Heading • 

School A School A School B City B City C City D 
Cohort 1 Cohort 2 



7 


Fall 


Q3 


Q3 


Q3 


Q3 


Q3 


Q3 


7 


Spring 


Q3 


Q3 


Q3 


Q3 


Q3 


Q3 


8 


Fall 


R3 


R3 


R3 


R3 


R3 


Q3 


8 


Spring 


R3 


R3 


R3 


Q3,R3 


R3 


Q3 ^ 


9 


Fall 


R4 


R2,R3,R4 


R4 


R4 


Q4 


R4 




Each of 


our six 


samples eoritaini 


5 Students who were 


tested 


at le 



the following five timeS! In fall and In spring of Grades 7 and 8 and 
In fall of Grade 9. These six samples do not contain all students who 
participated In the programs since they consist only of students with 
five. test scoresv Thereforei we compared the longitudinal samples to 
larger samples of all two-year participants who were tested at least ^ 
twice. These comparisons are presented in Appendix E, Appendix F con- 
train s'^the^ Call f o rn l^a^^d a t aT^^^par aft'e 1 y^B^ 

The grade-equivalent means and standard deviations for the samples 
are presented in able 6, The sample siaes range from 52 to 153* In 
Table 7j we consider Grades 7 and 8 separately, and we present for each 
estimates of achievement rates based on two different time periods. 
Column I contains estimates of Grade 7 achievement rates based on the 
traditional pretest (fall) to posttest (spring) interval. For all pro- 
grams and cohortSj these rates are quite high, ranging from 1,7 to 3,1 
months per month* Each rate clearly exceeds the month-for-month standard* 
Using this rate, all programs would be considered successful. 

Column II contains estiraates of Grade 7 achievement rates based on 
the interval from fall of Grade 7 to fall of Grade 8* These rates are 
all substantially below those found In Column II, ranging from 0,4 to 
1*3 months per month. However, three rates still exceed the month'-for- 
month standard. Using the rates in Column II, the reading program in 
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Table 6 

CALIFORNIA DEMONSTRATION PROGRAMS MEANS AND STANDARD DEVIATIONS 
IN GRADE EQUIVALENTS FOR THE CTBS 



Means 





7 Fall 


7 Spring 


8 Fall 


8 Spring 


9 Fall 


Mathetnatics ! City A 












School A 












Cohort 1 


5.4* 


6.7 


6.1 


7.7 


7.4 


(n - 52) 


(1.8) 


(2.2) 


(2.2) 


(2.6) 


(2.6) 


Cohort 2 


5.1 


7.1 


6.2 


7.4 


7.2 


(n ^ 67) 


(1.6) 


(2.3) 


(2.3) 


(2.4) 


(2.2) 


School B 


6.0 


8.2 


7:3 


8.7 


8.7 


(n ^ 153) 


(1.7) 


(2.2) 


(2.2) 


(2.6) 


(2.3) 



Reading . 

City B ■ 4.3 3.6 5.5 6.5 6.2 

(w-«-99) ^ ^(^^^^(^i^Q^^^(^:^,.^^(^:)^^^.^(^8h 

City C 4.6+ 5.8 5.0 7.5± 5.8 

(n = 54) (1.8) (2.0) (2.0), (2.4).*. (2.4) 

City D 5.7 6.9 6.3 7.3 7.8 

(n » 47) . (2.6) (2.7) (2.7) (2.8) (3.0) 



The grade-equivalent means ware Galculated by first nomputlng means in 
the expandad standard score scale. The standard score means were then 
converted to assoclatad grade-equivalent scores. The standard davla- 
tions were calculated In a similar faahlon, , 

^S in ce standard scores were not available at this tast administration, 
the means ware computed in grade equivalents at the Individual level, 

We are suspicious of this high score and have reason to believe that 
it may reflect unusually permissive testing conditions. 
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Table 7 



CALIFORNIA DmONSTRATION PROGRMS MONTHLY ACHIEVmENT RATES 
BASED ON TWO DIFFERMT PERIODS OF TIME* 



Grade 7 Grade 8 

X ^ 

7 Fall to 7 Fall to 8 Fall to 8 Fall to 

7 Spring 8 Fall 8 Spring 9 Fall 

Mathernatlcs : City A 
School A 

Cohort 1 1.9 0. 7 2/3 1.3 

(n ^ 52) 

Cohort 2 2.9 1.1 1,7 1.0 

(n ^ 67) 

School B 3.1 1.3 2.0 1.4 

(n^ 153) 



Reading 



City B 1*9 1.2 1*4 0.7 

(n ^ 99) 

City C 1. 7 0.4 3.6 0.8 

(n ^ 54) 

City D 1. 7 0. 6 1.4 1*5 

(n - 47) 



The rates are calculated by dividing the achievement gain for each 
interval (based on the means in Table 6) by the number of grade- 
equivalent months in the interval (7 for Columns I and III, 10 for 
Columns II and IV). 
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City B would be still judged successful, as would the mathematics pro* 
gram for Cohort 2 in School A and in School B. The remaining three 
programs would not be judged successful during the longer^ 12-month 
Interval. 

Columns III and IV contain estimates of Grade 8 achievement rates 
based on the traditional f all-tD»spring interval and the longer fall-to- 
fall intervals respectively. As beforej the achievement rates based on 
fEll to spring (Column III) are uniformly highj ranging from 1,4 to 3,6 
months per month. Using the rates In Column Ills all progrms would be 
considered qutte successful. As with all our ahalyseSs the rates based 
on fall to fall (Column IV) are substantially lowers ranging from 0.7 
to 1.5 months per month. Using these rates , two of the programs would 
not be considered successful. However^ in four of the programs the 
lower rates still exceed the 1,0 srandard and they would be considered 
successful. 

In Table 8, present monthly achievement rates for two-year evalu- 
ations based on the three different time periods described above (see 
Flgure'Dp Columns lA and IB are identical to Columns I and III in 
.Tab 1 a^7 --ani^give^ th e^^chiev em en.t_xa. heg, fn r^-e ach— pr& tes.t^t o^p o g 1 1 eg _ _^ _ 
interval. These rates are uniformly highs all easily exceeding the 
mon th-for-month standard. Programs with such high rates of achievement 
for two consecutive years would clearly be judged quite successful. 

Achievement rates based on the Interval from the beginning of 
Grade 7 to the end of Grade 8 are presented in Column II, These rales 
are also highs ranging from 0.9 to 1.7 months per month. Though these 
fates are lower than those calculated for each school year separately^ 
they are sufficiently high for the programs to still be considered suc- 
cessful, 

" Column III contains the achievement rates based on the 24-manth 
Interval. from the fall of Grade 7 to the fall of Grade 9, With one 
exception (City D), for each sample the rate in Column III is lower 
than both the rate in Column I and the rate In Column II. The achieve- 
ment rates fnr five of the six samples equal or exceed the mon th-f or-month 
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Table 8 



CALIFORNIA DS40NSTRATI0N PROGRAMS TWO-YEAR MONTHLY ACHIEVmENT RATES 
BASED ON THREE DIFFERENT PERIODS OF TIME* 



^ lA IB II III 

Fall to Spring 7 Fall to 7 Fall to 

Grade 7 Grade 8 8 Spring 9 Fall 

" -Ma^th_ema ties : City A 

School A ■ - -■ 

Cohort 1 1.9 2.3 1.4 10 
(n - 52) 

Cohort 2 2.9 1.7 1.4 1.1 
(n - 67) 

School B 3.1 2.0 16 1 A 
(n M 153) 

City B 1.9 L.4 1.3 1.0 



City C 1.7 3.6 1*7 0.6 

(n ^54) 

City D 1.7 1.4 0.9 1.1 

(n m 47) 



The rates are calculated by dividing the achiavement gain for each 
Interval (based on the means in Table 6) by the number of grade- 
equivalent months in the Intarval (7 for Columns lA and IB, 17 for 
Column I, and 20 for Column III), 



standardj with one considerably higher (School Bj City A), The rate for 
the sample from City C is substantially below the standard. In spite 
of the overall lower rates under this time frame, if an evaluation were 
based on an actual two-year interval, five of the six samples would still 
lead evaluators to judge the programs as successful. 

Again, we argue that the third time period^ spanning two calendar 
years j is the niost appropriate for a two-year evaluation* The other 
time periods do not consider the eKtent to which achievement is sustained 
during the sunmier months. Since some of the achievement is not sustainedj 
the two-year period generally yields lower rates. Yet, for five of the 
six samples this estimate of sustained achievement is still substantially 
higher than the expected achievement of 14 months over two years. Hence, 
five samples represent programs that would be judged successful even 
under the most demanding time interval. 




33 



22 



C 



V CONCLUSIONS 



We have analyzed several seti of data, all of which demonscrate that 
astlmates of achievement rates can differ dramatically when different time 
periods are used for an evaluation of a compensatory education program. 
Specif icallys we find that including the sunmer months in an evaluation 
often considerably reduces estimates of achtevement and hence monthly 
achievement rates* In other wordsj achievement gains made during the 
school year are not sustained^ even-until the next fall. These findings 
are consistently supported by the data regardless of the standardised 
achievement test used^ the grade levels the subject area^ or the program. 
While these data were not selected to be nationally representative of 
compensatory programs^ the consistency of the findings across different 
tests; grades, subjects , and programs suggests that our conclusions are 
general liable* 

We conclude that if compensatory education programs are going to be 
evaluated on-the basis of achievements it is essential to have measures 
of sustained achievement* We suggest that an evaluation of a one-year 
program be based at a minimmn on fall-to-fall achievement gains, iKtend- 
ing the argument to evaluations of multi-year programs, we suggest that 
such evaluations be based on a time period that Includes the summer inter- 
val following each year of the program. By Including the summer following 
the program in an evaluation, the judgment of program success will at 
least partially reflect the extent to which achievement is sustained and 
therefore will provide more useful information to those concerned with 
Improving programs. We also urge that a spring test be administered 
so that school^year and sunfflner achievement can be compared* This can 
provide a basis for future research on the eKtent and causes of sutraner 
losses, which can in turn lead to appropriate remedies. 

Many school districts already administer achievement tests to compen- 
satory education students in both the fall and the spring of each year* 
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The results of these tests can be used to create longitudinal data, such 
as we have analysed in this report. Although the longitudinal data will 
not be complete for all program participants, those students for whom 
no data are missing can provide a basis for estimating sustained achieve- 
ment. Hence, for these districts we do not reconroend any change in 
their collection of data* rather^ we raconmiend a modification in the way 
the data are used* In addition to estimates"of f all- Co-spring achieve- 
ment, these districts should estimate fall-to-fall achievement. 

We also do not recommend any change In the collection of data in 
those districts that administer achievement tests in the fall only. 
Although the school-year and sununer gains (or losses) cannot be separated, 
sustained achievement can be estimated by analyzing data from all students 
with at least two consecutive fall scores, (Xn this case, biases caused 
by^ attrition will be more difficult to Investigate. The type of analyses 
dona In Appendix B rely on also having data from achievement tests admin- 
Isterad In the spring.) 

In the school districts that administer achievement tests only in 
the spring, we do recomiend changes in the collection of data. A spring- 
to -spring -gain -cons 1st S"=^of two=^compo^ 

probably, loss) made prior to participation in the program and the fall- 
to-sprtng gain made while in the program. Therefore, the spring-to- 
sprtng gain does not provide any Indication of the extent to which the 
fall-to-spring, school-year gain is sustained. We recomnend that these 
districts also administer tests each fall, so that estimates of sustained 
gains can be made. - 

In conclusion, we urge that districts administer achievement tests 
minimally each fall and preferably each fall and spring. These data 
would provide the capability for estimating the extent to which school- 
year gains are sustained through the following summer. Both fall and 
spring tests have the added advantage of allowing a separation of 



Comparisons of the longitudinal sample with other samples will be required 
to investigate possible biases. See Appendices B and E, 
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school-year and suiraner achievement. Although this inforniatlon is not 
critical for estimates of annual gainSt it is valuable for studying the 
extent and causes of summer losses. If, for example^ the phenomenon Is 
a fuction of the measures used, the standardised achievement tests, one 
would want to change the measures not the prograin. If it is a? result 
of Instructional techntques that mitigate against retentiooj then the 
techniques should be changed. Since there are no simple solutions (for 
example, there Is little research to support the notion that surmner 
school would alleviate the summer losses)/ it is important to be able 
to determine why the losses occur in order to develop appropriate remedie 



25 

36 



BIBLIOGRAPHY 



Angoff, W, , "ScaJ.es, Norms, and Equivalent Scores," in R, U Thorndike, 
_ ed,, EducatiQn al Measurement (second edition, Merican Council on 
Educatton, Washington, D,^C,, 1971), 

Cronbach, L.,J,, Essentials of Pa^cholo g ieal Testing (third edition. 
Harper and Row, New York, 1970), 



Harcourt Brace Jovanovich, Development and Use of th e Grade Equivalent 
Scale (The author. New York, 1970), 



Tallmadge, G, K. and D, P. Horst, "A Procedural Guide for Validating; 
Achievement Gains in Educational Projects," RMC Report No. UR-240, 
RMC Research Corporation, Mountain View, California (1975), 

Thomas, T. and S, Pelavln, "Patterns In ESEA Title I Reading Aehievamant,". 
EPRC 4537-12 Stanford Research Institute, Menio^Fa-rk, California 
(March 1976). - ^ 




U.S. General Accounting Office, "Report to the Congress by the Controller 
General of the United States, Assessment of Reading Aetivitias 

Children" (Office of Education, Department of Health, Education, 
and Welfare, Washington, D, C, , 1975). 



37 



26 



ERIC 



^ppendiK A 

A COOTARISON :0F PATTERNO OF AGRiEVlMENT 
GAINS IN THREE MEJ nICS 



Od 




Appendix A 

A COMPARTSON OF PATTERNS OF ACHIEVEMENT GAINS IN THREE METRICS 



The data from the California D^^nstratlon. Programs cohtalned Compre^ 
hens Ive Test of Basic Skills (CTBS) raw scores for each student. The raw 
scores permitted us to perform our analyses in any metric we chose. In 
the body of the report, we used grade-equivalent scores in order to be 
consistent with previous studies of the impact of compensatory education 
on achievement* The purpose of this appendix is to present evidence In 
support of our contention in Section III that the patterns of Interest 
are not seriously affected by the metric. 

Table A-1 presents the grade-equivalent means, the standard score 
means^ and the percentile associated with each standard score mean for 
samples of students who were tested In the fall and spring of Grade 7 
and in fall of Grade 8, For each of the sampleSj the three metrics show 
the same patt ern- the mean subsXanJi^ftliy^^ln^^^ 

of Grade 7 and decreases from spring of Grade 7 to the fall of Grade 8, 
The similarity in patterns Is also reflected in Columns I and II of 
Table A-2. Columns I and 11, respectivelyj present the mean fall-to- 
spring and fall-to-fall achievement gains. No matter which metric Is 
useds the fall-to-spring mean gain is larger than the fall-to-*fall gain* 
The last column in Table A-2 presents the ratio of the fall-to-fall gain 
to the fall-to-spring gain* This ratio is the proportion of the fall- 
to-spring gain that Is sustained during the sutraner. (It the entire gain 
were sustalnedj the ratio would be 1, ) All three metrics demonstrate 
the smne flndlngf that a large portion of the fall-to-spring gain is 
not sustained over the surraner* This finding Is most pronounced when 
the ratio is expiressed in percentiles. The ratios of gains in grade 
equivalents and standard scores are quite iimllar* 
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Table A-1 



GRADE EQUIVALENTS MD STANDARD SCORE MEANS AND THE PERCENTILES 
ASSOCIATED WITH THE STANDARD SCORES FOR THE CTBS 
FOR STUDENTS TESTED 7 FALL, 7 SPRING, AND 8 FALL^ 



7 Fall 7 Spring 8 Fall 

Mathematics 
City A 
School A 



Cohort 1 GE 


5.1 


6.4 


5.8 


(n = 109) SS 


413 


459 


439 


Percentile 


14 


26 


15 


Cohort 2 GE 


5.3 


7.3 


6.4 


(n = 82) SS 


422 


495 


463 


Percentile 


18 


42 


23 


School B GE 


5.9 


8.1 


7.1 


(n s 241) SS 


445 


523 


489 


Percentile 


27 


56 


34 



Reading 



city B 


GE 


4.5 


5.7 


5.6 


(n 


= 102) SS 


405 


453 


446 




Percentile 


11 


19 


15 


City C 


GE 


4. 6 


5.8 


5.1 


<n 


= 65) SS 


415* 


456 


427 




Percentile 


14 


20 


10 


City D 


GE 


5.5 


6.5 


6.0 


(n 


= 81) SS 


444 


478 


459 




Percentile 


' 23 


• 28 


19 



Since standard scores for individuals were not avallablei the 
means was calculated in grade equivalents and translated Into 
a standard score. 
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Table A-2 



GRADE 7 ACHIEVEMENT GAINS IN GRADE EQUIVALENTS, STANDARD 
SCORES, AND PERCENTILES, AND THE MT 10 OF THESE GAINS" - 
FOR STUDENTS TESTED 7 FALL, 7 SPRING, AND 8 FALL - 



Math 
City A 
School A 
Cohort 1 
(n - 109) 



GE 
SS 



Parcentile 



Achievement Gain 



III 



I II \ Ratio of II 

Fall to Spring Fall to Fail (II I) 



1.3 
46 

12 



0.7 

26 
1 



0.54 
0.57 
0.08 



Cohort 2 GE 
(rT- 82); S_S_ 
Percentile 



2.0 

73 
24 



1.1 
41 

5 



0.55 
0.56 
0.21 



School B GE 
(n = 241) SS 
Percentile 



2.2 
78 
■ 29 



1.2 
44 

7 



0.55 
0.56 
0.24 



Reading 

City B GE 
(n =102) SS 
Percentile 



1.2 

48 
8 



1.1 
41 
4 



LO.92 
\0.85 
0.05 



City C GE 

(n * 65) SS 

- ; P € P e r c en t i 1 e 



1.2 
41 

6 



0.5 
12 
-4 



0.42 
0V29 
-o; 67 



City D GE 
(n = 81) SS 
Percentile, 



1.0 
34 

5 



0.5 
15 
-4 



0.-5 
0.44 

-0;80 



41 
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Appendix B - ; - - ^ 

THE REPRESENTATIVINESS OF THE CITY M 8AOTLES ■ \ 

The purpose of this appendix Is to determine if the City M samples 
presented In the text were representative of all students who had been 
in the program for at least one year. To determine thiss we formed sam- 
ples of one^year participants eonsisting of all students tested in both 
fall and spring but not tested the following fall* Hence these samples 
contain all tested students who were definitely in the program one year, 
but who were not included In the longitudinal samples*" We then compared 
the fall and =;spring means of the matched^ longitudinal samples from the 
text to the corresponding means of the samples of one^year particlpantSs 
These comparisons are shown in Colimins I and IT of Table B-1, The matchedj 
longitudinal samples are labeled M-L and the samples of one-year partici- 
pants are labeled 1-Y. 

In all gradeSi the means for the one-year participants are higher 
than those for the longitudinal samples. However, we are primarily con- 
cerned with these differences only to the extent that they suggest that 
the patterns of achievement (school-year and swnmer) are different for 
the two samples* For all grades except Grade 3, the means are equally 
higher In both fall and spring, hence the school-year (fall-to-spring) 
gains are quite similar for the two samples (see Column TV) .^^^^^ 
testrdirectly the summer achievement for the two samples since the one- 
year participants do not have the second fall test. Since the school- . 
yeargains are similar, we would expect summer gains (losses) to be 
similar* But it is conceivable that students with higher mean scores, 
such as the one-year participants, might show a different achlevementr> 
pattern over the summer. 

To investigate the possibility that there Is a difference In summer 
losses, we formed subsmples of each matchedj longitudinal samplep These 
subsamples consisted of only the highest scoring 60 percent to 80 percent 



Table B-1 



MEANS ACHIEVEbffiNT GAINS FOR THREE SA^ffLES 



Meang A ch 1 e veman t G a ins 



7 



jrade 


N 


I 

Pall 


11 
Spring 


III 
Fall 


IV 
Fall to 
Soring 


V 

Fall^to 
Fall 


M-L 


272 


2.23 


3.29 


2.78 


1.06 


0.55 


1-Y 


76 


2.63 


4.07 




■ 1.44 




M-L' 


177 


2.58 


3.68 


2.97 


1.10 


0.39-1 


M-L 


931 


2.65 


3.58 


3.18 , 


0.93 


0.55 


1-Y 


496 . 


2.93 


- J. 91 




0.98 




M-L^. 


705 


2.91 


3^. 82 


3.35 


■ 0.91 


0.44 


M-L 


980 


3.26 


4.30 


4.01 


1.04 


0.75 


1-Y 


582 


3.53 


4.51 




0.98 




M-L' 


801 


3.53 


4.55 


4.26 


1.02 


0.73 


M-L 


316 . 


3.85 


4.78 


4.42 


0.93 


0.57 


1-Y 


1141 


.4.28 


5.17 ■ 




0.89 




M-L' 


245 


4.27 . 


5.09 


4.70 


0.82. 


" 0.43 - 


M-L 


128 


4.35 


5.25 


4.95 


0.90 


~ 0.60 


1-Y 


318 


4.63 


5.54 




0.91 




M-L' 


112 


4.61 


5.48 


5.18 


0.87 


0.57 



B-3 

44 



of each matchedj longitudinal sample, based on the first fall scores. 
By including -only , the higher scoring students, the initial fall means 
; of the subsamples were all %^ithin 0,05 grada-equlvarent years of the fall 
mean 'scores of the 1-Y samples. Given similar fall scores, we assume 
that if the^subsamples have the same school-year gainsj they will have 
similar summer losses- Hence t^e first compare their achievement gains 
during the school year. The means and achievement gains for the sub- 
samples, labeled M-L'p appear in Table B-l» A comparison of l^Y with 
M*L' shows that In all caseSj with the exception of Grade 3, the differ- 
ence between the spring mean scores is less than 0*1 grade-equivalent 
years. Therefore, during the school year the tj^o groups show exactly 
the same pattern of achievement gains. 

If students who score higher in the first fall and spring have a 
different achievement pattern over the summer, then it should be apparent 
in the M-L- subsamples* We therefore compare the school -year and suiraner 
gatns of the higher scoring M-L' samples to the H-L samples* From Column 
IV, Table B-1, it is evident that the school-year gains and simmier losses 
are remarkably similar for the two sets of samples. This similarity is 
shown graphically in Figure B-1. Figure B-1 presents the gains by grade 
for each of the three samples described above* Each chart demonstrates 
the similarity of the fall test point for the one-year participants 
(1-Y) and the higher scoring, longitudinal subsamples (H-L^), With the 
exception of Grade 3, the school-year gains for these two samples are 
also quite similar* Each chart also demonstrates the slmllari. ty between 
both the school-year gains and sunmer'riosses for the entire matched, 
longitudinal samples (H-L) and the higher scoring subsamples^.^Jb}"Lf)* 

In suiranary, bur first analysis demonstrates that the school -year 
.gains are quite similar for the.matched, longitudinal samples (M-L) and 
the one-year participants excluded from these samples (1-Y)* The second 
"analysis Investigates whether this similarity also holds for the summer 
losses* We found that the higher scoring students In the longitudinal 
samples (M-L^), selected to match the fall .icores of the one-year partici- 
pants samples (l^Y), have spring scores and hence school-year gains" 
virtually idehttcar to the one-year participants. We also demonstrated 




for the higher scoring students in the longitudlnai ■ 
/: . s the seme as those for the entire matched, longitudlnal^^^ \ ; / 

samples (M-L), We conclude that If we had the second fall score for the ; 

. one-year participants not included in the longitudinal samples, they ■ ■ ' ' -'"^'/'^Jl 

^^^^^^^^ ; w the same pattern of summer losses* . . -r ■ ..z^.: 
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Appendix C 

PUBLIC SCHOOL STUDENTS IN CITY M BY COHORT 

Appendix C con teins additional information on the public school 
students in City M, Table C-1 describes : the cohorts for which we have 
data by year and grade level. Each cohort represents a wave of students 
arid is assigned. the Rumber that was the student's grade in 1971-72. Thus, 
we have two years of data for Cohort 1^ three years of data for Cohort 2, 
and four years of data for Cohorts 3-6. 

Table C-2 presents the means and standard deviations for the samples 

■defined., by. public, school stud en ts^with at least- three consacutlv rtest 

points for each cohort. The totalSj which are averaged across cohorts^ 
are the figures presented in Table 1 in tha text. Table C-3 corresponds / 
Co Table 2 in the text *and presents the gains and rates over two time 
periods, separately for each cohort. 

Tables C -4 and C-5 are analagous to C-2 and C-3 but are based on the 
sample defined by public school students with at lea^st five consecutive 
test points. The totals in Table C-4 correspond to the figures in Table 3 
in the text. Table C-5 presents the achievement rates based on three 
time periods for each cohort, and corresponds to Table 4 in the text. 



Table C-1 
NAMES OF COHORTS BY YE^" BY GRADE 



Grade 1971-72 
3 



1972-73 1973'-74 1974-75 



4 

5 
6 
7 
8 
9 



Cohort 3 
Cohort 4 
Cohort 5 
Cohort 6 



Cohort 2 Cohort 1 

Cohort 3 Cohort 2 V Cohort 1 

Cohort 4 Cohort 3 . Cohort 2 

Cohort 5 Cohort 4 Cohort 3 

Cohort 6 Cohort 5 Cohort 4 

Cohort 6 Cohort 5 
Cohort' 6 




Table G-2 



MEAHS AND STANDARD DEVIATIONS IN GRADE EQUIVALENTS 
- FOB THE GATES -Ma cGlNITIE READING TEST BY COHORT FOR 
PUBLIC SCHOOL STUDENTS WITH AT LEAST THREE CONSECUTIVE TIST POINTS 



Grade 3 - fcohort 1 " .152 

i t 

..... f . . ... 
I" 

bohoxt 2 90 

t . ..... 

Cohort 3 30 

Totals A 111 

s 

Grade 4 - Cohort 2 387 
Cohort 3 270 
Cohort 4 274 
Totals 931 

Grade 5 - Cohort 3 401 

Cohort 4 278 

Cohort 5 301 

Totals 980 

Grade 6 - Cohort 4 92 

Cohort '5 136 

Cohort 6 88 

Totals 316 

Grade 7 - Cohort 5 73 
Cohort 6 55^ 
- Totals 130 



Fall 






1.97 


3.05 


2 , 60 . 


(0.45) 


(1.10) 


(0,78) 


2.67 


3.72 


2.82 


(1-57) 


(1.86) 


(0.64) 


2.20 


3.26 


3.54 


(0.71) 


(0.96) 


(1.84) 


2.23 


3.29 ^ 


: 2. 78 








2.54 


3.60 


3.16 


(0.65) 


(1.04)' 


(0.88) 


2.81 


3.63 


3.07 


(1.13) 


(1.48) 


(0.88) 


2.64 


3.51 ■ 


3.31 


(0,67) 


(1.07) 


(1.13V 


2.65 


3.58 


3. 18 






i,U. ye) 


3.25 


4.45 


4.10 


(0.90) 


(1.30) 


(1.32) 


3.46 


4.52 


3.99 


(1.22) 


(1.62) 


(1.22) 


3.09 


3.89 


3.92 


(0.83), 


(1.14) 


(1.34) 


3.26 


4.30 


4.01 




\km Jo I - 


^ i . JU J 


3.59 


4.81 


4.41" 


(0.99) 


(1-36) 


(1.J2) 


4.08 


4.79 


4.45 


(1.38) 


(1. 64) 


(1.31) 


3.78 


4.75 


4. 38 


(1.02) 


. (1.29) 


(1.43) 


3.85 


4. 78 


4.42 


(1*20) 


(1.47) 


(1.32) 


4.37 


5.30 


4.84 


(1-19) 


(1.58) 


(1.34) 


4.33 


5.18 


5.10 


(1.32) 


(1.82) 


(1.50) 


4/35 


5.25 


4.95 


(1.24) 


(1.68) 


(1.41) 



Table ' - 

ACHlKVKMaiT GAINS MD MONTHLY RATES BASED ON TOO 
DIFFERENT PERIODS OF TIME FOR PUBLIC SCHOOL STUDENTS 
WITH AT LEAST THREE CONSECUTIVE TEST POINTS 



Aghievement In Grade- " " Monthly 

lEgutvalent Monthi^ Achievement Rates 



I II III ^ IV 

c y^.l t.c Spring Fall t© Fall Fall to Spring Fall to Fall 

— Ji.^ • ____ g 7) ^ (II ^ 10) 

Grade 3 

Cohort 1 152 _ 10.8 6.3 1,5 0.6 

Cohort 2 90 ^^ 10.5- r.5 ~ 1.5 - - 0.2 

Cohorts 30 10.6 13.4 "1.5 _ 1„3 

Total 2:^ \ L0.6 , 5.5 _ 1.5 0,6 

Grade 4 - 

Cohort 2 387 ■(> 6 6.2 1.5 0.6 

Cohort 3 270, L^.j 2.6 1-2 ^ 0.3 

Cohort 4 274 8.7 6.7 1.2 0.7 

Total 931 9.3 5,3 1,3 0,5 

Grade 5 . ■ - . .-^ ;'.-.■•>. 

Cohort 3 401 12.0 8.5 ^ 1. 7 0;8 

Cohort 278 10.6 '5.3 1.5 0.5 

^Cohort 5 . 301 . : 8.0 8.3 1.1 0,8 

Total 980 10.4 7.5 1.5 0.8 

Grade 6 - •■ - • ■ ••• 

; Cohort 4 92 12.2 8.2 1.7 0,8 

■-Cohort 5 136 ' 7.1 3,7 1.0 - 0.4 ^ 

Cohort 6 88 9.7 6.0 1.4 • 0.6 

Total 316 9,3 5.7 _1,3 0,6 

Grade 7 • , . 

Cohort 5 - 73- 9.3 4.7 1,3 0.5 

Cohort 6 55 8.5 7.8 1.2 0.8 

Total 128 9.0 6.0 1,3 0.6 



-Achievement is based on Che means in Table C-2 
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Table C-4 



MEANS mD standard! DEVIATIONS IN GRADE EQUIVALENTS ■ 
^ FOR THE GATES -MacGINlTIE READING TEST BY COHORT 

FOR PUBLIC SCHOOL STUDENTS WITH AT. LEAST FIVE CONSECUTIVE TEST POINTS 



I 

Pall 1 



: II 
Spring 1 



III 
Fall 2 



IV 

Spring 2 



V 

Fall 3 



Grade 3-4 



Cohort 2 


65 


2. 


73 


. 3.80 


2. 


88 


3.99 


3 


51 






(1. 


67) 


(1.96) 


(0. 


67) 


(1.11) 


(0. 


90) 


Cohort 3 


22 


2. 


12 


3. 10 


3. 


38 


4.51 


3. 


I9 






(0. 


60) 


(0.82) 




84) 


(2.07) 


(6. 


99) 


Totals 


87 


2. 


57 


3.62 


3. 


00 


4.12 


3. 


53 






(1. 


49) 


(1. 77) 


(1. 


10) 


(1.42) 


(0. 


92.) 



Grade 4-5 



Cohort 3 


178 


2.83 


3.65 


3.04 


4.29 


3.85 






(1.13) 


(1.49) 


(0.84) 


(1.28) 


(1.17) 


Cohort 4 


146 


2,66 : 


" ■ 3.41 


3.19 


4.24 


3.89 






(0.63) 


- (0.96) 


(0.92) 


(1.19) 


(1.12) 


Totals 


324 


2.75 


3.54 


3,11 


4,27 


3.87 






(0. 94) 


(1.28) 


(0.88) 


(1.^24) 


(1.15) 


Grade 5-6 














Cohort 4 


50 


3.39 


4.33 


"P 3.57 


4.89 


4.29 






(1.43) 


(1.86) 


(1,01) 


(1.40) 


(1.16) 


Cohort 5 


80 


3.11 


4.02 


4,04 


4,76 


4. 48 






(0.92) 


(1.16) 


(1.32) 


(1.58) 


(1.30) 


Totals 


130 


3.22 


4.14 


3.86 


4.81 


4.41 






(1.15) 


<l-^7) 


(1.22) 


(1.51) 


(1,25) 



Grade 6-7 



Cohort 5 


45 


4. 


08 


4. 


91 


4. 


52 


5 


44 


. 5 . 04 






(1. 


19) 


(1. 


58) 




07) 


(1 


47) 


(1.25) 


Cohort 6 


18 


3, 


76 


4. 


17 


4. 


16 


5. 


19 


4.98 






' cv. 


04) 


(0. 


99) 




46) 


(1. 


99). 


(1.08) 


Totals 


83 


3, 


99 


4. 


70 


4. 


42 


5. 


37 


5.02 






(1. 




(I- 


45) 


(1. 


17) 


(1. 


61) 


(1.19) 



Table C-5 



, TWO-YEAR MONTHLY ACHIEVEMENT RATES 
iaASED ON THREE DIFFERENT PERIODS OF TIME 
FOR PUBLIC SCHOOL STUDMTS WITH AT LEAST FIVE CONSECUTIVE TEST POINTS 



A B 
Year 1 Year 2 



II ' III 



Fall I to Fall 2 to Fall 1 to Fall 1 ,to 
.S£M£._ _N_, , Spring 1_ Fall 2 Spring 2 . Fair 3 



Grade 3-4 

Cohort 2 65 1.5 (10.7) 1.6 (11.1) 0.7 (12. G") 0.4 (7.8) 

Cohort 3 22 1.4 (9.8) 1.6 (11.3) 1.4 (23.9) 0.7 (14.7) 

Total 87 1.5(10.5) 1.6(11.2) 0,9(15.5) 0.5(9.6) 

Grade 4-5 

• Cohort 3 178 1 . 2 (8. 2) 1 . 8 (12-^54 0. 9 (14. 6) 0. 5 (10.2) 

Cohort 4 14S 1.1 (7.5) 1.5 (10.5) 0.9 (15.8) 0.6 (12.3) 

Total ,324 1.1 (7.9) 1.7(11.6) 0. 9 (14. 6) 0. 6 (11. 2) 

Grade 5-^ . 

Cohort 4 50 1.3 (9.4) 1.9 (13.2) 0.9 (15.0) 0.5 (9.0) 

Cohere 5 80 1.3(9.1) 1.0(7.2) 1.0 (16.5) 0.7 (13.7) 

Total 130 1.3 (9.2) 1.3 (9.5) 0.9 (15 .9) 0. 6 (11. 9) 

Grade 6-7 ^. 

Cohort 5 45 l-2=-t8.3) 1.3 (9. 2) 0.8 (13. 6) 0.5 (9. 6) 

Cohort 6 18 0.6 (4.1) 1.5 (10.3) 0.8 (14.3) 0.6 (12:2) 

Total 63 1.0 (7.2) 1.3 (9.5) 0.8 (13. 8) : 0. 5 (10. 3) 
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Appendix D 
STUDENTS IN CITY M NONPUBLIC SCHOOLS 



Appendix D contains the data for the nonpublic school students In 
City M* These tables correapond to those presented in the text and in 
Appendix C for the public school students. The cohort names are the 
same as those presented in Appendix Cp Table C-1, The reading program 
was not begun in the nonpublic schools until 1972-73*^ Theref ore. Tables 
D-1 through D-6 span fewer grades than the corresponding tables for the 
public school students. Table D-1 presents the means and standard devia^ 
tions (for the nonpublic school students with at least three consecutive 
test points) for each cohort. The achievement gains and rates over two 
time periods for these students are given in Table Tables D-3 and 

D-4 contain the same data as D-1 and D-23 respectively^ averaged over 
the cohorts, ^ 

Tables D-5 and D-6 present data for the non-public school students 
with at least five consecutive test administrations. Table D-5 contains 
the means and standard deviations for these students and Table D-6 presents 
monthly achievement based on three time periods, ^ 



Table D-1 



MEANS i^D STANDARD DEVIATIONS IN GRADE EQUIVALENTS FOB THE 

GATES-MacGINITIE READING TEST BY COHORT FOR NONPUBLIC 
SCHOOL STUDENTS WITH AT LEAST THREE CONSECUTIVE TEST POINTS 



_N_ JFall Spring Fall 



Grade 4 



Cohort 2 69 3,-34 ^ 4.40 4.09 

1 (0.84) (1.40) (1.29) 

Cohort 3 78 2.95 3.75 3.77 

(0.76) (1.00) (1. 12) 

Grade 5 

Cohort 3 89 3.66 5.01 4.54 

(0.99) (1.14) (1.13) 

Cohort 4 88 3.66 4.75 4.74 

(1.01) (1.44) (1.36) 

Grade 6 

Cohort 4 49 4.33 5.13 4.76 

(1.08) (1.28) (1.34) 

Cohort 5 68 4.43 5.14 5.19 

(1.30) (1.53) (1.50) 

Grade 7 

Cohorts 38 4.73 5.68 5.60 

(1.28) (1,88) (1. 61) 

Cohort 6 36 5.21 5.61 5.68 

(1.46) (2.21) (1.52) 
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D-3 



Table D-2 

ACHIEVraOIT GAINS MD MONTHLY RATES BY COHORT BASED ON TWO DIFFERENT 
PERIODS OF TmE FOR NONPUBLIC SCHOOL STUDmTS 
WITH AT LEAST THREE CONSECUTIVE TSST POINTS 



Grade 4 
Cohort 
Cohort 



2 

3 ^ 



^9 
78 



Achi€ 



t In Grades 
It Montha* 



i Fall to Spring 



10.6 



II 

Fall to Fall 



7,5 
8.2 



Monthly 
Achievement Rates 
III IV 
Fall to Spring Fall to Fall 
(1-^ 7) (II > 10) 



1,5 
1.1 



0.8 
0,8 



Grade 5 



Cohort 
Cohort 



89 
88 



13,5 
10,9 



8.8 
10.8 



1.9 
1. 6 



0.9 
1.1 



Grade 6 

Cohort 4 
^ Cohort 5 



49 

68 



8.0 

7 a 



4.3 
7.6 



1. 1 
1.0 



0.4 
0.8 



Grade 7 
Cohort 5 
Cohort 6 



38 
36 



9*5 
4.0 



8,7 
4.7 



1.4 
0. 6 



0.9 
0.5 



* ■ 

The achievement is baaed on the means in Table D-1. 



Table D^-S 



MEMS AND STANDARD DEVIATIONS IN GRADE EQUIVALENTS 
FOR THE GATES -MacGINIT IE READING TEST BY GRADE 



FOR NONPUBLIC 


SCHOOL 


STUDBSTS 


WITH AT 


. LEAST 


THREE 


CONSECUTIVE TEST 


POINTS 






I 


II 


T T T 
ill 


Grade 


N 


Fall 


Spring 


f a J. J. 


4 


147 


3.13 


4.06 


3.92 






(0.82) 


(1.24) 


(1.21) 


5 


177 


3.66 


4.88 


4.64 






(1.00) 


(1.30) 


(1.25) 


6 


117 


4.39 


5.14 


5.01 






(1.21) 


(1.42) 


(1.44) 


7 


74 . 


4.96 


5.65 


5.68 






(1,38) 


(2.05) 


(1.56) 



Table D-4 

ACHIEVEMENT GAINS AND MONTHLY RATES BY GRADE BASED ON TWO DIFFERENT 
PERIODS OF TIME FOR NONPUBLIG SCHOOL STUDMTS WITH AT 

-LEAST THREE CONSECUTIVE TEST POINTS ; 



Achlavement in ^ - 

Grade EqulvalenE-MQnths^ Monthly Achievement Rates 

Grade N Fall to Spring Fall to Fall Fall to Spring Fall to Fall 

; ; / ^^^^ (II -H- 10) 



4 


147 


9.3 


7. 9 


1.3 


0,8 


5 


177 


12.2 . 


9.8 


1.7 


1.0 


6 


117 


7.5 


6.2 


l.l 


0.6 


7 


74 


6.9 


6.8 


1.0 


0.7 



The achievemant is based on tha means in Table D-3* 



Table D-5 



ME^S MD STMDARD DEVIATIONS IN GRADE EQUIVALENTS 
FOR THE GATES-MacGINITIE READING TEST BY COHORT FOR 
NONPUBLIC SCHOOL STUDENTS WITH AT LEAST FIVE CONSECLmvE TEST POINTS 



JL Fall 1 Spring 1 Fall 2 Spring 2 Fall 3 



Grade 4-5 



Cohort 3 59 2.84 3.62 3.61 4.96 



4.45 



CO. 75) (0.93) C0.99"> (1,16) (1.12) 



Grade 5-6 



C»f>ort 4 39 3.38 4.49 " 4.43 5.14 4 74 

(0.98) (1.29) (1.08) (1.22) (1.'24) 



Grada 6-7 



Cohort 5 28 4.36 4.88 4.85 5,67 



5.75 



(1.35) (1.31) (1,31) (2,08) (1.44) 



Table D-6 

TWO-YEAR MONTHLY ACHIEVEMENT BASED ON THREE DIFFERENT 
TIME PERIODS FOR NONPUBLIC SCHOOL STUDENTS 
WITH AT LEAST FIVE CONSECUTIVE TEST POINTS* 



A B 

Year 1 Year 2 



II III 



Fall 1 Fall 2 Fall 1 ; : Fall L . 

J. to Spring 1 to Spring 2 Co Spring 2 to Fall 3 



Grada 4-5 

Cohort 3 59 i 1.1 (7.8) 1.9 (13.5) 1.2 (21.2) 0.8 (16.1) 
Grade 5-6 

Cohort 4 39- " 1.6 (11.1) I.O (7.1) 1.0 (17,6) 0.7 (13.6) 

Grade 6-7 : " ■ ■ ^'- ^ 

Cohort 5 28 0.7(5.2) 1.2(8.2) 0. 8 (13, 1) 0. 7 (13. 9) 



The achtevement in months is In parentheses.- This achievement is based 
on the means In Table D-5, 
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Appendix E 



THE REPRESMTATIVMESS OF THE CALIFORNIA S^LES 



The purpose of this appendix is to determine if the California sai 
pies presented in the text are representativa of all students ^ho had 
participated In the progrMs during both Grade 7 and Grade 8, To form 
samples of two-year participants, we included all stdciv ats who were tested 
in the fall of Grade 7 and in the spring of Grade 8, the fall of Grade 9, 
or the spring of Grade 9. We then compare the fall and spring means of 
the smples of two-year participants to the means of the matched, longi- 
tudinal sample from the text: This comparison is shorn in Table E-1. 
The samples from the text are labeled M-L and the sample of two-year 
participants are labeled City C is not included because we did 

not have access to the necessary data. Of the five samples presented, 
the five means are almost identical for two of the samples* City A, 
School B and City B. Therefore we conclude that the longitudinal samples 
for these two groups are representative of the larger samples. For Cohort 
2 of City A, School A, the first four means are virtually identical for 
the two samples* but the Grade 9 fall mean is'lower for two-year partici- 
pants (sample 2 -Y). Since this results in an even more precipitous sum- 
mer drops the difference does not affect our conclusions. 

For Cohort 1 of City A^ School A, and City D, the means for the two 
samples i^re quite different. For Cohort 1, the first four means are con- 
sistently lower for the two-year participants, indicating that the longi- 
tudinal sample is biased. The patterns of achievement are still similar 
for the two groupsr large gains over the school year and a large loss 
over the summer. However, the direction of the difference is reversed 
for the Grade 9 fall test point. Since the 2-Y sample size is reduced : 
considerably between Grade 8 spring and Grade 9 fall, this may not be 
surprising. For City all five means are consistently lower for parti- 
cipanti (a range of four to six months). Since these differences are 



consistent across all five test points, the patterns of interest are not 
affected. 

Overall^ we conclude that the longitudinal samples are representa- 
tive of Che larger sample with the possible exception of Cohort 1 in the 
fall of Grade 9, Although the means differ somewhat^ the pattern of gains 
over the school year and losses oyer the summer are similar for the larger 
samples. 



Table E-1 

A COMPARISON OF MEMS FOR TOO S^LES 







7 Fall 


7 Spring 


8 Fall 


8 Sprlns 


9 Fall 


N 


Math 
















City A 
















School A 
Cohort 1 


- M-L 
2-Y 


5.4 
5.0 
n=l05 


6.7 
6.4 

n«99 


6. 1 
5.6 
n=93 


7,7 
7,0 
n=97 


7.4 
7,5 
n=67 


52 


Cohort 2 


- M-L 
2-Y 


5.1 

5.1 
n=102 


7.1 
7.0 
n=95 


6.2 
6.2 
n*84 


7.4 
7.5 

n=85 


7.2 
6.7 

n=86 


67 


School B 


M-L 
2-Y 


6.0 
5.9 
n=217 


8.2 
8.1 

n-206 


7.3 
7. 1 
n=214 


8.7 
8.7 
n=208 


8.7 
8.7 
n=167 


153 


Read Ing 
















City B 


M-L 

2-Y 


4.3 
4.3 
n=ll6 


5.6 
5.6 
n=105 


5.5 
5.4 
n=lll 


6.5 
6,5 
n=112 


6,2 

■%.2 ) 
n=114 


99 


City C 
















City D " 


M-L 

■ 2-Y 


5.7 
5.2 
n=89 


6.9 
6.3 
n^85 


6.3 
5.7 

n=82 


7.3 
6.7 
n=82 


7.8 

7.4 
n^55 


47 



63 
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Appendix F . 

; , CALIFORNIA STUDENTS AND FALL ' 

Tables F-1 through F-4 presant additional data In irade- equivalents 
for the California Demonstration Projects. Table F=l presents the means 
and standard deviations for students who were tested in the fall and 
spring of Grade 7 and the fall of Grade 8. Table F-i gives the achieve- 
ment gains and rates for each sample based on the means in Table F-1. 
Tables F-3 and F-4 correspond exactly to Tables F-1 and F -2 and are based 
on the students who were tested in the fall and spring of Grade 8 and 
the fall of Grade 9, 
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Table F-1 



MEMS MD STMDARD DEyiATIONS IN GRADE EQUIVALENTS FOR THE CTBS 
FOR STUDENTS TESTED 7 FALL, 7 SPRING, AND 8 FALL 



N 7 Fall 7 Spring 8 Fall 



Mathemktics 
City A 
School A 



Read IriR 



at the individual level. 



Cohort 1 


109 


5.0 


6.2 


5.6 






(1.7) 


(2.2) 


(2.2) 


Cohort 2 


82 


5.2 


7.3 


6.3 






(1.6) 


(2.2) 


(2.2) 


School B 


241 


5.8 


8. 1 


7,1 






(1.6) 


(2.0) 


(2.0) 



City B 


102 


4.3 5.6 


5,4 






(1.7) (1.8) 


(1.8) 


City C 


65 


4.6* 3 . 7 


4.9 






(1.7) (2.0) 


(1.9) 


City D 


81 


5.4 6.'4 


5.8 






(2.2) ^ (2.6) 


(2.6) 


* 








Since Standard 


Score 


s were not available 


at this test 


administration, 


the' 


mean was computed in 


grade -equivalents 



Table F=2 



GRADE 7 ACHlEVMENT GAlNS MD MONmY RATES BASED ON 
TWO DIFFERENT PEEIODS OF TIME FOR STUDmTS 
TESTED 7 FALL, 7 SPRING, AND 8 FALL 



Achievement In ' 
Grade-Equtval^nc Mgnthg* Jjonthly Achievement Rates 
I _ II.- Ill IV 

-M^ Fall to Spring Fall to Fall Fall to Sprlnp Fall to Fall 



Mathematics 
City A 
School A 
Cohort 1 
Cohort 2 
School B 



109 
82 
241 



12 
21 
23 



6 
11 
13 



(I + 7) 



1*7 
3,0 
3,3 



(II + 10) 



o;6 

1.1 
1.3 



Reading 
City B 

City C 

City D 



102 
65 
81 



13 
ll 
10 



11 

3 
4 



1.9 
1.6 
1,4 



1.1 
0.3 
0.4 



Based on means in Table F^l, 




Table F-3 



MEMS AND STANDARD DEVIATIONS IN GRADE EQUIVALENTS FOR THE CTBS 
FOR STUDENTS TESTED 8 FALL, 8 SPRING ^ MD ? FALL 



N 8 Fall 8 Spring 9 Fall 



Mathematics 
City A 
School A 



Cohort 1 


68 


5.8 
(2.2) 


7.4 
(2.6) 


7.3 
(2.4) 


Cohort 2 


89 


6.0 
(2.2) 


7.2 
(2.4) 


6.9 
(2.2) 


School B 


187 


7.1 
(2.2) 


8.6 
(2.4) 


8.6 
(2.2) 


Reading 










•'City B 


107 


5.5 
(1.7) 


6.5 
(1.8) 


6.2 
(1.8) 


City C 


64 


5.0 
(1.9) 


7.5* 
(2.4) 


5.8 
(2.4) 


City D 


64 


6.2 
(2.6) 


7.1 
(2.8) 


7.6 
(2.8) 


* 

We are susplclou 
to believe that 


s of this high 
it may reflect 


score and 
unusually 


have reason 
permissive 



testing conditions. 
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Table F-4 . . ' 

GHADE 8 ACHIEVmENT GAINS MD MONTHLY/ RATES ; BASED ON TOO ' 
DIFFERENT PERIODS OF TIME FOR STUDENTS TESTED 

, 8 FALL, - 8 SPRING, MD : 9 FALL . - ' - ^.^ /.^ 

Achievement tn 

^ Grade = Equivalent Month Monthly Achievement Rates 

_ I II ~III ' IV 

N Fall to SprlnE Fall to Fall Fall to Soring ^ Fall to Fall 

; ' (I 7) (11^ 10) 



- Mathematics— 
City A 
School A 

Cohort 1 68 
Cohort 2 89 

, School B . 187 

Reading 

City B _ : 107 

City C 64 

City D 64 



16 
12 

15 



10 
25 
9 



15- 
9 

15 



7 
B 
14 



2.3 
1,7 

2.1 

1.4 
3. 6 
1.3 



1,5 
0.9 

1.5 



0,7 - 
0,8 

i;4 



Based on 



in Table F-3 



... ---.-f: 
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